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A Criterion For Counseling’ 


Cuirrorp P. FROEHLICH 
University of California, Berkeley 


INTRODUCTION 


O NE of the most obvious objectives of 
counseling is to increase the coun- 
selee’s knowledge of himself. This objec- 
tive is justified on the grounds that a 
client must know himself if he is to make 
satisfactory adjustments to the difficulties 
which brought him to counseling. This 
justification is particularly pertinent 
when the problems of educational and 
vocational planning are considered. A 
client must have a reasonably accurate 
estimate of his mental ability before he 
can intelligently decide whether to enter 
college or to pursue a professional career. 
Likewise, he needs some knowledge of his 
interests before he can logically choose 
a curriculum or a vocation in harmony 
with them. The word “before” has been 
italicized in the preceding sentences to 
highlight a significant point in connec- 
tion with this investigation which studies 
a criterion of counseling effectiveness. 
The criterion compares one perception of 
self with an external measure of self, 
and notes agreement prior to counseling. 
A second comparison of self-perception 
and external measure is made after coun- 


*This study was aided by a grant from the Re- 
search Fund of the Department of Education, 
University of California. The cooperation of 
Robert Brownlee, Principal, Demonstration Sec- 
ondary School, made it possible to conduct the 
study in his school. I. Aileen Poole assisted in 
the collection of the data. E. Wayne Wright and 
Joseph Martin helped in the analysis of the data, 
with the use of the Computer Laboratory of the 
University. 


seling, and agreement again is noted. The 
criterion is the change in agreement be- 
tween the first and second comparison. 
The criterion studied combines good 
features of both sociological and psycho- 
logical criteria. The sociological aspects of 
the criterion are, of course, found in the 
postcounseling behavior of the client. He 
cannot score favorably on the criterion, 
except by accident, unless he has in- 
creased self-knowledge. Hence, an im- 
provement in such knowledge creates a 
“before” situation, subsequent to which 
the client is more able to show improved 
behavior. 

The criterion in this investigation 
might be viewed also as a psychological 
one. It differs, however, from the Q sort 
and similar criteria used by Rogers and 
others in that it is based on an objective 
measure, a test score. Rogers (8) simply 
compared one perception of self with an- 
other perception of self in order to note 
changes. The present study compares 
self-perceptions obtained in the form of 
self-ratings with objective test data. 


Terms Used in this Study 


For the sake of brevity, certain terms 
will be used to denote concepts which 
would require fuller explanation if the 
following definitions were not made. The 
terms “congruence” and “agreement” are 
used interchangeably. They are used to 
indicate coincidence between two or 
more measures. “Score” represents the re- 
sulting number when the individual's 
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actual test score is converted (in the man- 
ner to be described) to its corresponding 
step on a five-point scale. “Score” means, 
then, the quintile rank-equivalent of the 
test score. The word “rating” is used to 
indicate the numerical step on a five- 
point scale which the individual checked 
when asked to rate himself. 


DESIGN OF STUDY AND COLLECTION 
OF DATA 


The purpose of this study was to in- 
vestigate a variable which appears to pos- 
sess the quality of being a sociological- 
psychological criterion. To accomplish 
this purpose, specific questions were 
formulated to be answered by the data 
collected in the experimental situation. 

1. The first of these questions was: Js 
there a significant difference between the 
agreement of precounseling rating with 
score and the agreement of postcounseling 
rating with score? A related question was 
also asked. Are the counselees’ changes in 
agreement in the direction of greater 
agreement? In other words, do the com- 
parisons made before counseling differ 
significantly from those made after coun- 
seling; and does counseling increase the 
congruence of self-ratings and scores? It 
was anticipated that if these questions 
were answered in the affirmative, the fol- 
lowing one could be asked of the data. 

2. Does the counseled group at the 
conclusion of the experiment show more 
ratings which are congruent with scores 
than the noncounseled group? This ques- 
tion formalized the expected differences 
between counseled and noncounseled 
groups of students. In contrast, the first 
question made explicit the idea that the 
number of agreements would change 
from before counseling to after counsel- 
ing. In this case, each counselee acted as 
his own control. In the other, compari- 
sons were made between groups of stu- 
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dents unlike in their participation in 
counseling. 


Locale ‘and Population of the Study 


The basic data for this study were collected 
during the summer of 1953 at the Demonstra- 
tion Secondary School operated by the Univer- 
sity of California in cooperation with the Oak- 
land, California, public schools. The summer 
high school completed its sixteenth session in 
1953, at which time the total enrollment was 951 
students. Sixty-nine per cent of the students were 
residents of Oakland, 30 per cent came from 
other California communities, and 1 per cent 
from outside of California. The students came 
from 58 high schools. Their median age was 14 
years, 11 months, and their median grade place- 
ment during the preceding semester was the sec- 
ond semester of grade 10. 

A poll of the students’ reasons for attending 
the summer school indicates that 27 per cent 
were there to repeat a subject or to improve 
the mark, 25 per cent to gratify special interests, 
and the remainder were completing requirements 
for college entrance or high school graduation. 
In comparison with national norms, the student 
population was skewed in the direction of su- 
perior ability on a test of mental ability. 

All students who during the preceding se- 
mester had been enrolled in grades nine through 
twelve were required to take a battery of tests 
during their so-called “guidance period.” This 
period was regularly scheduled but carried no 
credit. The students, after testing was completed, 
were not expected to be at school during their 
guidance period. 

The purpose of the testing program was ex- 
plained to the students at the first testing session. 
They were told that they were participating in 
an experimental testing program which was con- 
ducted for research purposes by the University 
of California. They were informed also that the 
test results would be available to them if they 
wished to talk with a counselor. No effort was 
made to induce students to make use of the 
counselors. At the first testing session, students 
were asked whether or not they wanted to talk 
with a counselor. The data in Table 1 indicate 
the number who wished to be counseled as well 
as the number who were actually counseled. The 
discrepancy between the total of g51 students 
enrolled in the school and the 485 in Table 1 is 
accounted for by a variety of factors. Some were 
excused from the testing program if their out-of- 
school employment made it necessary, as were 
also foreign students whose knowledge of English 
was not sufficient to complete the tests. Approxi- 
mately 200 students were eliminated from the 
study because they had not been enrolled in a 
high schoo] during the preceding semester. 
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TABLE 1 
STUDENTS INCLUDED IN THE STUDY 


Male Students 
Do you wish to be 


Female Students Total 


counseled? 


Not 
Counseled Counseled 


Not 
Counseled Counseled 


Not 
Counseled Counseled 


Yes 113 63 
No 9 63 
Did not answer question 3 21 


107 
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Others were eliminated because their criterion or 
test data were incomplete. In all, 466 of the 951 
students were not used in the present investiga- 
tion. 


The Test Instruments 


The following tests were used: SRA 
Primary Mental Abilities; Intermediate 
(9); Kuder Preference Record-Vocational, 
Form C (6); Test of Mechanical Compre- 
hension, Form BB (1); and SRA Youth 
Inventory, Form A (7). 


The tests of interest, mental ability, and me- 
chanical comprehension were selected because 
they were judged to be typical of those used in 
high school counseling programs, Studies of their 
reliability and validity have been reported in 
the journal literature and publishers’ manuals. 
For each of these tests a listing of references, as 
well as several critical reviews, are included in 
the three most recent editions of Buros’ meas- 
urement yearbooks (2, 3, 4). All appear to be 
sufficiently reliable for the purposes of this 
investigation. 

The validity of the tests used in this investiga- 
tion deserves a general comment, Although they 
are generally accepted as being valid instruments, 
in one view, their validity may be considered as 
unrelated to the study. In this instance the study 
is viewed as dealing with the question of whether 
counseling increases the congruence between the 
counselee’s self-rating and his score. In other 
words, does counseling teach a person his score so 
that he does reproduce it in the form of a self- 
rating. It is not a question of whether the score 
on a test of, say, outdoor interests is a valid meas- 
ure of outdoor interests. The fact that the 
counselor accepts the score as an indication of 
outdoor interests and interprets it as such to 
the counselee lends credence to the idea that if 
counseling is effective counselees should show 
more agreement between their self-ratings and 
scores on tests of outdoor interests after counsel- 
ing than they did before. 

The criterion used in this study is concerned 
with the observed change when the agreement 


of precounseling score and ratings is compared 
with the agreement of postcounseling scores and 
ratings. Essentially the criterion is anchored to 
agreement of score with rating, rather than 
agreement of the rating with some absolute or 
perfect measure of the characteristic upon which 
the counselee rates himself. If in counseling the 
score is interpreted as being indicative of a more 
perfect measure of the characteristic (whether 
or not it actually is is beside the point) then the 
criterion should be influenced in the hypothe- 
sized manner implied in the two major questions 
stated previously. 

The validity of the SRA Youth Inventory is 
not germane since this check list was not used as 
one of the variables in the criterion. It was em- 
ployed, rather, in an attempt to identify differ- 
ences between persons who ranked high and 
those who ranked low on the criterion. In a 
sense, an attempt was made to validate it against 
the criterion. 

All the raw scores on the tests were con- 
verted to quintiles on the basis of the published 
norms with the exception of the mechanical 
ability test. Here no appropriate norms were 
available, hence quintiles were based on a 
sample of students making up the population 
used in this study. 


The Ratings 


The self-ratings were secured by means 
of a scale designed for the study. Students 
were asked to rate themselves on a five- 
step scale for each of the ten interest 
areas measured by the Kuder Preference 
Record, and also their mental and me- 
chanical ability. The interest or ability 
to be rated was described in nontechnical 
language. 

Students were asked to complete the 
rating scale at the beginning of the first 
testing session. In the last week of the 
summer term, seven weeks after students 
completed the first rating scale, a sec- 


3 
8 36 17 99 
| 16 8 37 


4 CLIFFORD P. 


ond one was administered. It was identi- 
cal with the first except that the instruc- 
tions were changed to include a rationale 
for the request for repeat ratings. The 
second ratings were made during regular 
class periods, 


The criterion investigated in this study was 
based upon these self-ratings. Such ratings have 
been utilized in a variety of research studies and 
have themselves, in some instances, been sub- 
jected to study. The literature pertinent to the 
topic “ratings” is voluminous; that pertaining to 
self-ratings is more limited. It was recognized 
that self-ratings had a limitation in terms of 
reliability. Other measures might have greater 
reliability but none seemed to hold as much 
promise for studying changes in self concept as 
did self-ratings. Indeed, the task of rating one- 
self requires one to reveal aspects of one’s self 
concept or to resort to some sort of deception 
in order to conceal it. This is a compelling 
reason for using changes in self-estimates as a 
criterion of counseling. But an even more 
pertinent reason is to be found in the very un- 
dependability of the ratings. One of the objec- 
tives of counseling is to eliminate or at least 
reduce the influence of factors which contribute 
to the unreliability of ratings. Counseling should 
improve the accuracy of the counselee’s self con- 
cept. If it does, it would make it possible for 
him to rate himself more accurately. 

At the outset of the study another limitation 
of the criterion was recognized: there is more 
involved in self-rating than just knowledge of 
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self. A counselee may, for example, “know” on 
the intellectual level that his performance on 
a test of mechanical comprehension is equiva- 
lent to the second centile of entering students in 
an engineering school. But, if he dreams of 
bridging the Bosporus, irrigating the Sahara, or 
inventing a space ship, he may not really believe 
his mechanical comprehension is limited. His 
concept of himself as a_ successful engineer, 
figuratively speaking, insulates him from the 
threat of accepting the meaning of his low me- 
chanical comprehension test score. The net result 
is that he rates his mechanical comprehension 
higher than his score rates him. The implica- 
tion of this limitation for this study is that the 
criterion is not a measurement of changes of 
actual self concept but rather is assumed to be 
a reflection of those changes. 

The reliability of the ratings used in this 
study was estimated from data collected in the 
same school situation but in a different summer. 
Students were asked to rate themselves and then 
a week later to repeat the process. No tests were 
given, nor were interviews held during the inter- 
val between ratings. The reliability coefficients 
reported in Table 2 were computed by using the 
customary formula for product-moment correla- 
tion and were corrected for coarse grouping in 
the manner described by Wert, Neidt, and 
Ahmann (10). All of the uncorrected coefficients 
are significant at the .o5 level except that for 
“persuasive.” Of the 13 corrected coefficients, 9 
are significant at the .o1 level. These coefficients 
seem to indicate that the ratings used in this 
study have a degree of reliability not unlike that 
reported in other studies of ratings. True, they 


TABLE 2 
RELIABILITY COEFFICIENTS OF RATINGS 


Correlation of Ratings 


Uncorrected 


Corrected 


Reliability 
Coefficient 


Level of 
Significance 


Level of 
Significance 


Reliability | 
Coefficient 


Interests:| 
Outdoor 
Mechanical 
Computational 
Science 
Persuasive 
Artistic 
Literary 
Musical 
Social Service 
Clerical 

Abilities: 
Mental 
Mechanical 


-40 


.O1 
.O1 


.05 
.O1 
.O1 
-05 
-16 
-O1 
.O1 
-O1 


05 


| 

| 

31 = 

31 -88 

31 -62 .66 

31 -39 205 

30 “25 -18 

31 -43 +05 -46 

30 -74 -O1 

31 -64 -O1 .68 

31 -O1 -51 

31 .38 +05 

27 .78 -OI 

30 46 .O1 

— 
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are not as high as might be desired. Nevertheless, 
as the writer stated earlier, ratings appear to him 
to have a potential for assessing self concept and 
this was an important factor in choosing them. 


The Counselors 


The persons who did the counseling in 
the Demonstration Secondary School 
were all graduate students specializing in 
student personnel and counseling psy- 
chology in the Department of Education, 
University of California. At the time of 
their participation in this study, the 
counselors were enrolled in the university 
summer session working toward ad- 
vanced degrees in counseling psychology. 
During the preceding semester nine had 
been employed as secondary school teach- 
ers, six as counselors at the college level, 
four as  teacher-counselors in high 


schools, three as full-time counselors at 
the secondary level, two as school social 
workers, one as teacher-psychologist, one 
as head counselor, and one as guidance 


supervisor. 


The counselors chose their own orientation to 
counseling. None was completely nondirective. 
All were aware of the fact that the pupils had 
been told first of the availability of counseling 
when they reported to the testing sessions. The 
promised opportunity to learn and talk over the 
test results was used undoubtedly by the ad- 
ministrators as a motivating factor during the 
series of five testing sessions. Hence, some 
pupils who reported to the counseling office 
came for the express purpose of test interpreta- 
tion. Counselors made an effort to identify pupils 
who had an inadequate notion of the kind of 
relationship that the counselor was willing to 
establish. They made a special effort to make 
certain that all counselees realized that the 
counselor was willing to talk about things other 
than test scores. 


The Criterion 


In counseling practice, as has been 
pointed out, one of the major objectives 
is to help the client improve his knowl- 
edge of himself. The assumption is made 
that for a client to make wise choices 
which involve him, it is necessary for 
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him to know himself. In some counselors’ 
orientation, the client can best discover 
self-knowledge in a nondirective or 
client-centered relationship. Other coun- 
selors consider it a more meaningful ex- 
perience if the counselor plays a more 
active role. Although counselors vary 
greatly in their relative adherence to 
either of these two points of view, there 
is rather general agreement among them 
that an important outcome of counseling 
is an increase in the client’s knowledge of 
himself. The criterion investigated in 
this study was designed to measure this 
outcome. 

The criterion was change in agree- 
ment between a person’s rating of his 
ability or interest and an objective meas- 
urement thereof. In short, the client’s rat- 
ings on a five-point scale in each area 
were compared with quintile ranks based 
on his test scores in corresponding areas. 
In terms of the desired outcome—im- 
provement in knowledge of self—certain 
assumptions had to be made concerning 
the criterion data. 

Although it was recognized that the test scores 
were fallible and perhaps might n~* be accurate, 
it was assumed that they did m. -t an image 
of the client. From this assumptica it followed 
that if the client had accurate knowledge of self 
his view of himself would correspond with the 
test-revealed one. Conversely, it was expected, 
therefore, that clients would learn about them- 
selves from test scores. 

A second assumption was made that if a 
person knew what his tested abilities and inter- 
ests were he would be able to rate them ac- 
curately, Conversely, if he did not know what 
his abilities were, then he would be unable to 
rate them accurately. 

The third assumption was made that al- 
though the subjects of this study would un- 
doubtedly have a considerable amount of in- 
formation about themselves, the process of test- 
taking and counseling would improve this knowl- 
edge. These three assumptions were basic to the 
design of the study. 

The data are presented for three groups 
of students: Counseled, counseled-with- 
out-test, and not counseled. 
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In the counseled group were included those 
students who were counseled and with whom the 
counselor discussed the results of the test. It 
should be noted that the counseled group varied 
from test to test; that is, not all persons who 
were counseled had all test results interpreted 
for them, One counselee, for example, may have 
had the interest inventory and mental ability test 
interpreted while another may have discussed 
the interest inventory and the mechanical apti- 
tude test with the counselor. Both counselees 
would be included in the counseled group for 
the ten scores of the interest inventory. The first 
counselee would be included in the counseled 
group for the mental ability test and in coun- 
seled-without-test group for the mechanical apti- 
tude test. Conversely, the second would be in- 
cluded in the counseled group of the mechanical 
aptitude test and counseled-without-test group 
on the mental ability test. 

The second group, counseled-without-test, in- 
cludes those persons who were counseled but 
for whom the particular test was not interpreted. 
Noninterpretation of tests was occasioned by 
several factors such as a delay in scoring the test 
so that it was not available at the time of 
counseling or the judgment of the counselor that 
the test score would not make a significant con- 
tribution to the counseling. 

The third group, “not counseled,” includes 
those individuals who took the tests, but who 
were not counseled. Counseling was available 
for all who desired it but no effort was made to 
induce students to seek counseling. Of the 116 
students who indicated at the first testing ses- 
sion that they did not wish to be counseled, only 
17 later changed their minds and sought 
counseling. 


FINDINGS 


The data concerning each test were 
separated into three groups according to 
whether or not the individual was coun- 


seled, counseled-without-test, or not 
counseled. In order to study the agree- 
ment between scores and ratings, each 
individual’s score and corresponding rat- 
ing were compared. If both fell on the 
same numerical step of the five-point 
scale, it was tallied as agreement. That is, 
if the individual's score was a “five” and 
his rating a “five,” the values were con- 
sidered in agreement. But if his score was 
“five” and his rating “four,” the values 
were tallied as disagreement. The agree- 
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ment between first rating and score was 
determined, as was also the agreement 
between second rating and score, Each 
individual on the basis of agreement be- 
tween score and ratings of each area 
was Classified into one of four categories, 
namely: 


Category 1. The score and first rating dis- 
agreed and so did the score and 
second rating. Persons in this 
category consistently rated them- 
selves inaccurately. 

The score and first rating agreed 
and so did the score and second 
rating. These individuals were 
consistent in rating themselves 
accurately, 

The score and first rating dis- 
agreed, but the score and the 
second rating agreed. Included in 
this group are those individuals 
whose first rating was incorrect 
but who correctly rated them- 
selves on the second rating. They 
changed from “disagree” to 
“agree.” 

The score and first rating agreed 
but the score and second rating 
did not. These persons changed 
their rating from an accurate one 
to an inaccurate one, They went 
from “agree” to “disagree.” 


Category 2. 


Category 3. 


Category 4. 


Percentages based on the data tabu- 
lated according to the above-described 
categories are shown in Table 3. It is ob- 
vious that the majority of individuals’ 
ratings consistently disagreed with their 
score. This does not mean that they did 
not change their ratings. They might 
have rated themselves too low on the first 
rating and too high on the second; in 
both instances score and rating disagreed. 
The median percentage for this category 
for the counseled group was 55, for the 
counseled-without-test group was 54, and 
for the not-counseled group was 61. 

These median percentages are indicative of a 
situation of vital concern to counselors. Why is 
it that the majority of pupils in this study were 
unable to rate accurately their abilities on even 


such a gross scale as the five-point one used 
here? In a preceding section it was pointed out 
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TABLE 3 
PERCENTAGES FOR CATEGORIES OF AGREEMENT BY GROUPS AND AREAS 


Counseled-Without-T est 


| 


128 


Dis- 
agree 
to 
Agree 
Ir 
08 
17 
Ir 
18 
20 
14 
15 
17 
23 


Io 
09 


that self-ratings were used because they required 
the subject to reveal aspects of his self concept 
or resort to deception. The data in this study 
cannot yield the reason for the inaccuracies. 
They do, however, hint at the possibility that 
lack of agreement between rating and test score 
was brought about not so much by a lack of 
“knowledge” of self as it was by a lack of 
“acceptance” of the knowledge or an unwilling- 
ness to reveal the knowledge through ratings. 
Consider, for example, the rating of mental 
ability. All of the subjects in the study had had 
extensive experience in school which provided 
many opportunities for them to acquire a sound 
basis for judging their mental ability. School 
marks, teacher statements, feelings of success 
and failure, parent judgments, and peer evalua- 
tions are illustrative of the experiences which 
would help a person acquire an accurate self 
concept. Prior to counseling, 79 per cent of the 
subjects inaccurately rated their mental ability. 
After counseling, when they would have been 
given the knowledge necessary to rate themselves 
accurately, 68 per cent persisted in rating them- 
selves inaccurately. 


Comparison of Interest Profiles 


The Kuder Preference Record—Voca- 
tional, Form C yields ten scores, each 
indicative of interests in a particular 
area. Counselors usually interpret indi- 
vidual scores in the light of the total pro- 
file. 

Thus, one student's highest scores might be at 
the 85th centile in the clerical and musical areas, 


while another student's highest scores are in 
the same area but are equivalent to centiles of 


60. In both cases, the counselor would probably 
interpret the highest scores as indicative of 
areas of greatest interest. But the interpretation 
in both would continue to the point of helping 
the client see the strength of his interests in 
relation to those of persons in the norm group. 
Essentially, profile interpretation is a process 
of discerning the strengths of interest in the 
ten areas measured by the Kuder in relation to 
each other and to the normative population. 
Because such interpretations were common prac- 
tice among the counselors included in this study 
it appeared desirable to consider the accuracy 
with which students reported their obtained 
Kuder scores when taken together as a profile. 
To make the profile comparisons, the data were 
reprocessed using a technique of assessing simi- 
larity between profiles (5). The technique is 
illustrated by the following data concerning 
one student: 


Kuder Interest Areas 


First rating 5 s.4 2 4 4 5 
Test score 3 4 5 2 2 s 5 


—2 0 0 2 


Difference 2 
Square of 
difference 4 4 0 0 1 4 


In the “o” (Outdoor Interest) column, the “5” 
is the student’s rating of his outdoor interest 
which he made before taking the test. The “3” 
is the quintile category based on his obtained 
test score. The “2” is the difference when the 
score is subtracted from the rating. And 
finally, the “4” is the square of the difference 
between score and rating. The data for the other 
areas were treated in a similar manner, A figure 


7 
Con- In Con- | Cis | | Agree Con- | Con. | Dis- | Agree 
est Co Dis- — | sist- | oe | Agr w | catly | Sist- | agree | to 
ree | Agree | Agree | agree agree | Asree | | agree | agree | Agree | Agree | agree 
Interests: | | 
Outdoor 175} 69 | 12 | 13 | 06 | 64 | | o8 
Mechanical 174] 55 17 13 4 z 14 14 16 
Computational 175} so | is | 37 | 38 | 17 | a30| | | | ts 
ence 17 4 
Persuasive 175 | 53 18 | 20 | 09 | 38 
Litera: 175 | 52 22 15 | 54 2 I 26 12 
Musical s3 | a | | | 20 | 229 | | as | | 
Social Service 175 17 19 3 16 12 II 
Clerical 175| 56 | 17 | 12 | 14 | | 
Abilities: 
Mental 159 | 68 15 11 06 17 | o8 | 218 | 66 ar ae 13 
6o | 15 | | 158| 62 | 13 | 229] 6x | 13 | 14 | 12 
57 10 1% | 67 16 212 | 64 16 | 08 
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of 18 is obtained by adding the numbers in 
bottom row above. The square root of 18, or 
4.243, was used as this student’s first-rating- 
profile score. 


Another profile score was obtained for 
this student by comparing the interest 
ratings he made after counseling with 
his test scores. The technique used fol- 
lowed the pattern described above. This 
process yielded second-rating-profile 
score. In the case of the student cited 
above, his second-rating-profile score was 
3.464. Similarly, the data concerning all 
other students in the study were proc- 
essed to yield for each student a first- 
rating-profile score and a second-rating- 
profile score. 

Then the second-rating-profile score 
was subtracted from the first-rating-profile 
score. In the case of the student used as 
an example in the preceding paragraphs 
this computation yielded a positive num- 
ber, .779 (i€., 4.243 — 3.464 = .779)- 
The algebraic difference between the pro- 
file scores indicated whether the indi- 
vidual had moved toward greater con- 
gruency between the profile of ratings 
and the profile of scores, or toward less 
congruence, A positive difference be- 
tween the profile scores indicated that 
the second profile of ratings corre- 
sponded more nearly to the obtained pro- 
file of scores than did the first profile of 
ratings. Conversely, a negative difference 
means that the first ratings were more 
consonant with scores than were second 
ratings. In fine, positive differences 
showed increased “accuracy” and nega- 
tive differences a deterioration in “ac- 
curacy.” 

Separate frequency distributions of the 
differences between profile scores were 
made for the counseled, counseled-with- 
out-test, and not-counseled groups. The 
distributions ranged from high positive 
through zero difference to high negative. 
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The zero-difference category was not in- 
terpreted as indicating that the student 
did not change his profile of ratings. The 
zero difference merely indicated that the 
subject’s second profile of ratings agreed 
neither less nor more closely with the pro- 
file of scores than did his first rating pro- 
file. 

For the distributions of the differences 
between the first-rating-profile score and 
the second-rating-profile score the means 
and standard deviations were computed. 
Table 4 contains these data. The means 


TABLE 4 


MEAN AND STANDARD DEVIATION OF 
DIFFERENCES BETWEEN PROFILE SCORES 


| Coun- 
seled 
With- 
out 
Test 


Not 
Coun- 
seled 


Coun- 


Statistic seled 


Mean of difference .26 

Standard error of 
mean 

Standard deviation 


Number of cases 


-41 -04 


.09 .12 
1.09 
175 


.07 
-97 
230 


66 


varied in magnitude in the expected 
manner; namely, the counseled group 
showed the most change in the direction 
of greater agreement, the not-counseled 
group the least change. A ¢ test indicates 
that the means of the counseled and the 
counseled-without-test groups are signifi- 
cantly different from zero. In consider- 
ing the mean difference of .41, one 
should not overlook the fact that the dif- 
ference between score and rating could 
not exceed 4.00. Hence the difference is 
slightly more than 10 per cent of the 
largest possible difference. To test the 
significance of the difference between 
means, ¢ ratios were computed. The ¢t 
ratio between the mean of the counseled 
group and the mean of the noncounseled 
group was 3.52, significant at the .o1 
level. The ¢ ratios between the mean of 


| 
| 
| 
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TABLE 5 


COMPARISON OF CERTAIN CHARACTERISTICS OF SUBGROUPS SELECTED ON BASIS OF ACCURACY 
OF RATING OF INTEREST 


Counseled 
Variable 


Counseled Without 
Interest Test 


Not Counseled 


Deteri- 


Increased erated 


Deteri- 
orated 


Deteri- 


Increased ented 


Increased 


Mean age 

Mean school grade 

Mean P.M.A. score 

Mean number of tests inter- 
preted 

Number of: males 


females 
Number: 
desired counseling 
did not desire counseling 


16.3 16.1 
10.6 10.3 
2.5 I 


3-5 
36 
26 


38 
24 


25° 
288 | 


23° 


* Some students did not respond to the question, ‘‘Would you like an opportunity to talk with a 
counselor?” Therefore, these totals differ from others in the table. In instances where the number of 
nonrespondents was excessive, tests of significance were not applied. 


the counseled-without-test group and the 
counseled and the not-counseled groups 
were found to be 1.03 and 1.63, re- 
spectively. Neither was statistically sig- 
nificant. 

The data thus far reported have an- 
swered affirmatively the two questions 
spelled out in the section which dealt 
with the purpose of the study. They re- 
vealed that there was a significant dif- 
ference between precounseling and post- 
counseling criterion measurements, The 
data also revealed that these observed 
differences were in the direction of more 
agreements after counseling. The sta- 
bility of these differences was also demon- 
strated by the differences between the 
counseled, counseled-without-test, and 
not-counseled groups. The neat stair-step- 
ping of mean differences in Table 4 from 
smallest for the non-counseled group to 
largest for the counseled group lends cre- 
dence to the belief that the observed 
changes are in part at least a function of 
counseling. 


Comparison of Factors 


In this section data will be presented 


concerning factors thought to be associ- 
ated with changes measured by the cri- 
terion. They are: age, grade placement, 
intelligence, sex, number of tests inter- 
preted, and desire for counseling. 

In order to determine if the factors 
were associated with the criterion, the 
subjects were divided on the basis of the 
difference between the first-rating-profile 
score and the second-rating-profile score. 
The 27 per cent of the students with the 
largest positive differences (increase in 
“accuracy”) and the 27 per cent of the 
students with the largest negative dif- 
ferences (deterioration in “accuracy’’) in 
the area of interests were identified in the 
counseled, counseled-without-test, and 
not-counseled groups. This process 
yielded the six subgroups. 

For each subgroup the statistics sum- 
marized in Table 5 were obtained. When 
appropriate tests of significance were 
applied to differences between the “in- 
creased” and the “deteriorated” sub- 
groups in the counseled, counseled- with- 
out-tests, and not-counseled groups, none 
of the differences was found to be sta- 
tistically significant. Comparable results 


9 
15.8 15.9 16.1 15-7 
10.4 10.6 10.7 | 10.3 
3°3 3°5 3-1 3-1 
23 24 II 
45 45 15 12* 


CLIFFORD P. FROEHLICH 


TABLE 6 
NUMBER OF PROBLEMS CHECKED ON THE SRA Youth Inventory 


Counseled 


Increased | Deteriorated 
N=34 N=29 


Counseled Without Test 


Increased Deteriorated 
N=14 N=12 


Mean| SD Mean | SD 
6.64 


Not Counseled 


Increased 


I A Deteriorated | 
nventory Areas N=ss | 


| ratio | t ratio 


Mean| SD Mean| SD | Mean 


4-34 3-87 | 4.92 -17 -20 | 7.25 6.44 


My School 


Myself 
Getting Along 
with Others 
My ae and 
amily 6.12 
Boy Meets Girl 4 3.93 
Health 2.29 
Things in 
General 
Basic Difficulty 
Score 


7.67 13.14 
6.54 8.86 


6.42 8.29 
4.36 
3-57 
2.57 


4-34 4-21 


11.24 | 14.14 


9.96 
7.80 


6.96 
5.11 


3-56 
3.05 


3 
9-25 | 7-43 | 1.27 . 
6.75 | 8.70 | .70 -45 
4.67 -59 


2.50 -05 
2.92 -54 
2.33 +25 


Ann 


4.00 -38 | 3.03 


15.05 


12.50 |13. -35 1.08 


were found 
mechanical 
treated. 

On the basis of these data, it was con- 
cluded that the improvement or deteri- 
oration in the “accuracy” of students’ self- 
ratings apparently was not a function of 
their age, school grade, mental ability, 
sex, desire for counseling, or number 
of tests interpreted during counseling. 
It appeared, therefore, that the criterion 
employed in this study will be useful in 
the further study of counseling because 
it was not influenced by gross factors 
over which counselors have little con- 
trol. This conclusion led to a desire to 
learn even more about factors associated 
with the criterion. One attempt to study 
such factors will be reported in the next 
section. 


when mental ability and 
aptitude were similarly 


Comparison of Problems 


The SRA Youth Inventory appeared 
to be an instrument which might be used 
to identify some of the more subtle 
characteristics which distinguished those 
students who show more agreements be- 
tween scores and ratings after counseling 
than before. The Inventory is a list of 
298 problems of which the testee checks 
those which apply to him. The more 
problems he checks the higher his score. 


Did students who showed more agreements 
have more or fewer problems than those who 
showed fewer agreements after counseling? The 
answer to this question was sought in a compari- 
son of the SRA Youth Inventory scores of 100 
students who showed the greatest increase in 
agreements with gg students who showed the 
greatest deterioration in the agreement between 
their profiles of interest scores and ratings. These 
students were drawn from the upper and lower 
27 per cents of the distribution of differences 
between first- and second-rating-profile scores 
previously described. The mean inventory score 
of the “increased” group was 46.92 with a 
standard deviation of 30.32. The “deteriorated” 
group had a mean inventory score of 41.72; the 
standard deviation was 29.64. The ¢ ratio for 
the difference between these means was 1.22, not 
significant. Hence, the question asked above was 
answered: there was no difference between the 
groups in terms of the total number of problems 
checked on the Inventory. 

The next step was to determine whether or 
not there was a difference between the “in- 
creased” and “deteriorated” groups in the num- 
ber of problems checked in specific areas. The 
SRA Youth Inventory, as previously pointed out, 
yielded scores in eight areas of adjustment. In 
order to further refine the analysis, the groups 
were broken down into counseled, counseled- 
without-test, and not-counseled subgroups. The 
mean number of problems checked by each sub- 
group for each area is presented in Table 6, In 
Table 6 the ¢ ratios based on the differences 
between means are also given. None was found 
to be statistically significant. Hence, it was con- 
cluded that the SRA Youth Inventory scores 
did not identify those students who showed 
more agreements between their profile of scores 
and ratings after counseling than before. 

The two comparisons based on number of 
problems checked did not take into account the 
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| 
SD | 
6.00 4-50 
| 7.56 | 6.77 -68 
ih | 5.87 | 1.07 
6.17 6.22 | .61 
3-29 -46 | 3.06 | 5.85 | .07 
3.81 -77 | 3-24| 3-55 | 
2.82 -14 | 2.65 | 2.32 +74 
| 
4-00 | | 3-55 | 4-74) 
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possibility that certain particular problems 
might be checked by the “increased” group and 
others by the “deteriorated” group. The previ- 
ously discussed analyses were based simply upon 
number of problems checked, Hence an item 
analysis was undertaken, The groups were di- 
vided into random halves. The resulting groups 
were (a) Increased I, N= 50; (b) Increased II, 
N = 50; (c) Deteriorated I, N = 49; and (d) De- 
teriorated II, N = 50. The item responses of the 
Increased I and Deteriorated I groups were 
tabulated and percentage of response for each 
group for each item was computed, If the differ- 
ence between the percentages for the two groups 
was significant at the .o5 level, the item was 
retained. 

There were 35 of the 298 items in the test re- 
tained. Of these, 32 were keyed for increase and 
3 keyed for deterioration, These items were cast 
into a scoring stencil and all answer sheets for 
the test were rescored. A single score was ob- 
tained by subtracting the deterioration-keyed 
items from the increase-keyed items. The mean 
scores and standard deviations were computed. 
These data are presented in Table 7. The differ- 
ences between the means of the Increased I and 
Deteriorated I groups yield a ¢t ratio of 7.58, 
highly significant. This was expected since the 
scoring key was based on these groups. In an 
attempt to cross validate, the 35-item scoring 
key was applied to the answer sheets of the 
subjects included in the Increased II and De- 
teriorated II groups. The data in Table 7 con- 


TABLE 7 


MEAN AND STANDARD DEVIATIONS ON SCORES 
DERIVED FROM ITEM-ANALYSIS KEY 


(Cross- Validation Sample) 


Stand- 
ard 
Devia- 
tions 


Group 


Increased I 
Deteriorated I 
Increased II 
Deteriorated II 


4.56 
2.20 
2.84 


3-54 


cerning these two groups indicated that the 35- 
item key did not hold up. When a ¢ test was 
applied to the difference between the means of 
these two groups, it yielded a ¢ ratio of .65, 
not significant. 


These analyses of the SRA Youth In- 
ventory data suggested that the increased- 
deteriorated classification was not a func- 
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tion of the number or kind of problems 
the student checked. 


SUMMARY AND DISCUSSION 


. The over-all purpose of this investiga- 
tion was to study a proposed criterion 
for the evaluation of counseling, namely, 
change in agreement between self-rating 
and score, It is appropriate to ask, “What 
was learned about the criterion?” De- 
tails in the answer to this question can 
be found in the preceding pages. They 
may be summarized briefly. The criterion 
data varied in the expected direction, 
that is, the findings followed the logic 
of improved self-knowledge after counsel- 
ing. From this point of view the cri- 
terion which was adopted appears to 
hold promise as a useful indicator of 
counseling effectiveness. 

Its potential usefulness was further 
substantiated by the findings that the 
criterion appeared not to be affected by 
age, sex, school grade placement, intel- 
ligence, or desire for counseling. These 
are factors over which a counselor has 
no control (except by limiting his prac- 
tice to specified types of clients). The 
important implication of these findings 
is that the criterion variable is independ- 
ent of such factors which are extraneous 
to the counseling process per se. 

A related finding was that neither the 
number nor the kind of “problem” 
which counselees verbalized on the prob- 
lem checklist was associated with the 
criterion variable. If this result were ac- 
cepted at face value, it would do vio- 
lence to beliefs which counselors have 
about their relative effectiveness with 
clients having “minor” as opposed to 
“major” problems of adjustment, or 
with clients having certain kinds of 
problems (e.g., vocational choice as op- 
posed to feelings of inferiority). In con- 
sidering these data, it is important to 


50 8.00 

50 2.51 

50 4.76 

49 5.18 
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keep in mind that the criterion involved 
ratings in the areas of interests and 
ability. It was not concerned with so- 
called problems of personal adjustment, 
the forte of the problem checklist used 
in this investigation. But even in areas 
of the problem checklist closely related 
to interests and ability, no statistically 
significant association with the criterion 
was noted. 

This brief synopsis of the findings of 
this investigation can serve as a starting 
point for considering next steps. 


The criterion explored in this investigation is 
an “immediate” one, i.e., it is available within 
a short time after the experiment is completed. 
But its relationship to the long-range objectives 
of counseling has not been established. Herein 
lies a great weakness. If one were to argue that 
self-knowledge is the object of counseling, the 
criterion studied might be presumed to be a 
reasonable measure. However, few counselors, if 
any, would be satisfied with self-knowledge as an 
end in itself. The common expectation is that 


FROEHLICH 


self-knowledge should influence counselee be- 
havior. Here the criterion is silent, for it does 
not measure behavior. And in this silence is 
found an important gap in the usefulness of the 
criterion, as well as a suggestion for additional 
research. In future research it might be possible, 
for example, to discover that the criterion is 
associated with more intelligent, more construc- 
tive, or more adaptive postcounseling behavior. 
If such a discovery were made, then the criterion 
would acquire sociological significance. 

Another fact must be considered. Some stu- 
dents who were not counseled showed as much 
change on the criterion in the direction of more 
“accurate” ratings as did students who were 
counseled. Conversely, some counselees de- 
teriorated in the “accuracy” of their ratings as 
much as some not-counseled students. What 
factors are associated with change as measured 
by the criterion? The design of the present ex- 
periment was such that only counseling as total 
process was studied. Are there variations within 
the process from counselor to counselor, coun- 
selee to counselee, or problem to problem which 
account for the observed differences in the 
criterion variable? If so, are these factors also at 
work in different form among not-counseled 
students? These are crucial questions for which 
no ready answers are available. 
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